1. Technical Field
Embodiments of the invention relate to establishing rules for data quality when extracting data from one or more data sources.
2. Discussion of the Related Art
When loading data from one or more data sources into a system for use in an application, the data must typically undergo some sort of data quality analysis and/or data cleansing to ensure the data is suitable for use with the application. Data quality requirements for the same data can differ based upon a particular application which will use the data (e.g., different clients and/or different departments of a client may have different requirements for data quality). However, while certain general data quality rules can be established that ensure data is provided having a certain format, the rules are typically set and established by logic associated with the application or at a system source that extracts data from a data source. To change some or all of the data quality rules typically requires a change in an application or source system code. Because of this, data quality rules typically cannot be modified at run time, but instead need to be evaluated during a data processing cycle, with changes only being made after the processing cycle.
Since specific rules that apply to one application may differ for another application, it is difficult to ensure precise data quality for a number of applications retrieving data from one or more data sources.